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Abstract. Teachers encounter immense variety in literary competence among students in their second- 
ary school classrooms. Yet, little is known about how they perceive and deal with this variety. Moreover, 
little is known about the literary development that students undergo during their school career and the 
different levels of development that can be identified. The aim of the present study is to design an in- 
strument that can help to describe relevant differences in literary competence between students and to 
specify the individual literary development process of students in upper secondary education (aged IS- 
IS). To tie in with teaching practice, we designed a research method by which we could explore the 
shared pedagogical content knowledge of a diverse group of six expert teachers. With the aid of ques- 
tionnaires and panel discussions, data was collected on the following question: what does a student 
with a particular level of literary competence demonstrate with a particular literary text? The data anal- 
ysis resulted in 14 indicators of literary competence which we then used to describe six successive com- 
petence levels. These levels are validated by both developmental theory and the practice of literature 
teaching. 

Key words: literary competence; literary development; pedagogical content knowledge; reading levels; 
text selection. 


1. INTRODUCTION 

Which literary texts are able to stimulate student literary development under 
which circumstances is one of the key questions in literature teaching (Beach, Ap- 
pleman, Hynds & Wilhelm, 2006, 2011; Nikolajeva, 2010). It seems to have become 
even more pressing since the 1990s, as numberous countries switched to a more 
student-oriented curriculum and as teachers seek to respond more effectively to 
the differences they encounter in their classrooms (Bonset & RijIaarsdam, 2004). 
This also happened in the Netherlands, where a major overhaul of education saw 
the introduction of a student-oriented curriculum in 1997 as the government made 
'taking account of differences' a priority area within the changes. The switch to a 
student-oriented approach had implications for the formal examination syllabus for 
literature education, with the emphasis moving to the literary development of stu- 
dents (De Kievit & Wagemans, 1996). For us, this revamp of education was the im- 
petus to begin a study of student literary development in upper secondary classes. 
Before examining these issues in more detail, we will first explain them within the 
Dutch context so that the reader can better understand this article. 


1.1 Dutch Delta 

In the Netherlands the school subject Dutch is made up of different subcurricula 
which have little to do with one another. The main emphasis is on the language 
arts and literature. Of these two subcurricula, which are taught by the same teach- 
er, the domain of literature allows for the greatest freedom. Teachers themselves 
decide which texts to work on, and decide which objectives to emphasize and how 
much time to devote to literature. This liberalism is characteristic of Dutch litera- 
ture teaching and has been the subject of on-going discussions about the scope, 
content and aims of literature teaching. Since the 1960s these discussions have 
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covered issues like how learners should interact with literary texts, what role liter- 
ary texts play in society and what learners need to know to become good, inde- 
pendent readers of literature. The aims changed over time in response to changing 
ideas about teaching the mother tongue in general (Griffioen & Damsma, 1978; Ten 
Brinke, 1978; Van de Ven, 2011) and in response to changes in the academic liter- 
ary domain (Van de Ven, 1996; Van Schooten, 2005; Verboord, 2005). Teachers 
moved from a focus on 'work, author and literary history' in 1950-60, to 'work as 
immanent text' (1970s), to 'work as reflection of society' (late 1970s) and to 'work 
as constructed by the reader' (late 1980s). 

Studies of teaching practice reveal that these different approaches have all left 
their mark on the landscape of Dutch literature teaching (Rijlaarsdam & Janssen, 
1996; Van de Ven, 1996; Janssen, 1998; Witte, Janssen & Rijlaarsdam, 2005; Van 
Schooten 2005; Verboord 2005). This diversity of approaches looks as a river delta 
in which 'mainstream' approaches flow alongside a host of more minor brooks and 
streams. In other words, there are just as many literature teaching curricula as 
there are teachers. This indicates the literature curriculum as one of the 'ill- 
structured domains' (Spiro, Feltovich, Jacobson & Coulson, 1991). Whereas well- 
structured domains are subjects where most researchers and teachers agree on the 
aims and structure, ill-structured domains are characterized by an unsystematic 
structure and a multitude of visions. 

Rijlaarsdam and Janssen (1996) have mapped this 'delta', identifying four 
main approaches: (1) cultural literacy, (2) aesthetic awareness, (3) social awareness 
and (4) personal development. Both the cultural-historical and literary-aesthetic 
positions have a long tradition and are well-established in classroom practice. Alt- 
hough many teachers have embraced the student-centred (or reader-centred) ap- 
proach since the late 1980s, they still have difficulties with its pedagogical opera- 
tionalization (Ravesloot, 1992; Janssen, 1998; Witte, 2008). Teachers are not 
trained in the reader-centred approach and also lack the resources. During their 
university studies in Dutch, and before that at secondary school, they are taught 
the dominant cultural-historical and literary-aesthetic approaches. School texts also 
emphasize this content (literary theory and history) and offer neither student nor 
teacher support in developing literary competence. School texts that took a differ- 
ent approach were used infrequently (Witte, 1994). 

1.2 Channelling the delta 

Freedom of education is enshrined in the Dutch constitution, giving teachers and 
schools a degree of autonomy. We use 'degree' with purpose, because there are of 
course rules attached to founding a school and the final central examinations in 
secondary education are regulated by the government. In addition, the govern- 
ment has attempted to channel the many different 'streams' in the Dutch educa- 
tional delta. In the 1990s the learning-to-learn paradigm became the focus of cur- 
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riculum design in Dutch education. 1997 saw a full-scale overhaul of the education- 
al system and the introduction of what is called the 'study house'. Study house is a 
metaphor for a pedagogic approach based on activity and independence. Sets of 
attainment targets and exam syllabuses were implemented to regulate this pro- 
cess. The curriculum for Dutch language and literature was divided up into four 
domains (reading skills, writing skills, oral skills and literary development) to be 
taught in parallel. The examination syllabus for literature comprises three subdo- 
mains: literary development (e.g. reporting personal reading experiences), literary 
theory (using theoretical terms to analyse and interpret literary texts), and literary 
history (presenting an overview of Dutch literary history) (Stichting Leer- 
planontwikkeling, 1996). A student's personal literary development was allocated a 
key place in the new examination syllabus for literature. Students were required to 
read an average of four literary works by Dutch writers a year and to report on 
their reading experience and appraisal in a reading log. Depending on the educa- 
tional track, teachers were advised to administer an oral exam on eight or twelve 
books based on the students' reading records, which contained their reading logs 
and reflections on their literary development. This led to changes in the content of 
the oral exams. Whereas discussion used to centre on cultural-historical back- 
grounds and the interpretation of a work, it now also covered the student's per- 
sonal reading experience and appraisals. As these personal discussions revealed the 
students' actual reading performance, teachers became even more concerned 
about how to encourage the development of their students' literary competence. 

1.3 Differentiation 

To gauge their students' literary starting level and attitude to reading, many teach- 
ers ask students at the start of upper secondary education (grade 10, age 15) to 
write a 'reading autobiography': an account about their attitude to reading and 
their development as a reader of fiction. There are big differences between stu- 
dents at this age, as the following passages from reading autobiographies show: 

Rutger (age 15, grade 10) 

'I don't read books anymore these days. I like to read specialist literature about agricul- 
ture, like The Farm, Harvest and The Tractor. But I do have to read books for school. I'll 
find that tough because my entire family simply hates reading books. My father has 
only ever finished three books. Just like me, at the moment. (...) I expect the reading 
record is going to be quite a tough task for me that will take up a lot of my time. I think 
it's a waste of time, but I'll probably learn something from it.' 

Cecile (age 15, grade 10) 

'Last year I read all the books by Arnon Grunberg. I like his style of writing. In some 
ways he reminds me of the way I write and think. I read Ronald Giphart on the advice 
of my friends, but I think his writing is a load of rubbish. (...) Last summer I wanted to 
read more books, different books. I got a pile of 'good reads for beginners' from my 
parents and my brother, and I read books by Franqoise Sagan, Willem Frederik Her- 
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mans and others. I then started reading What is literature? by Jean Paul Sartre. Grun- 

1 

berg, Hermans and Mulisch are my favourite writers at the moment.' 

Teachers wishing to encourage the development of literary competence of all their 
students must be able to differentiate and must know the zone of proximal devel- 
opment of their students. This means knowing which literary texts and reading ac- 
tivities will help a student progress to a higher level (Vygotsky, 1978; Bandura, 
1986; Schunk 2000). However, differentiation seems to be a difficult teacher's skill; 
international studies show that few teachers master this skill (Hattie, 2009; 
Kyrakides, Creemers & Antoniou, 2009). Dutch teachers are no different from their 
counterparts in other countries. All PISA reports (2000, 2003, 2006, 2009) have 
shown that Dutch teachers achieve relatively good results for the middle group of 
students, but that they fall short when it comes to weak and very good students. 
Apart from the practical problem of the heavy teaching load in Dutch secondary 
schools (a fulltime job means 26-29 lessons a week and 8 to 10 groups of 25 to 32 
students), there is a cognitive problem. Teachers do not have an adequate mental 
frame of reference for observing, labelling and classifying differences between stu- 
dents, let alone being able to identify and label the different stages of development 
(Schunk, 2000; Witte, 2008; Hattie, 2009). It is against this background that we be- 
gan our research into an empirical basis for a literary frame of reference that 
teachers could use to distinguish different levels of literary competence. 

1.4 Shared pedagogical content knowledge 

Research reports on the success or failure of educational change often show that 
the implementation of research findings is unsuccessful if it does not tie in with the 
experiences, knowledge and views of teachers (Kennedy 1997; National Research 
Council 2002). This is something we wish to avoid in our study. An important pre- 
requisite for successful implementation is that the outcomes should be recogniza- 
ble to teachers (Van de Ven 1996; Kennedy 1997; Verloop, Van Driel & Meijer 
2001). This is why we have taken the shared pedagogical content knowledge of 
teachers as our starting point (Shulman 1986). Shulman defines this type of practi- 
cal knowledge as 

'... that special amalgam of content and pedagogy that is uniquely the province of 
teachers, their own special form of professional understanding' (Shulman, 1986: 8). 

To formulate a theory about student literary development in upper secondary edu- 
cation, we will take as our starting point the pedagogical content knowledge that 
teachers share concerning different levels of literary competence. This knowledge 
will enable us to gain an understanding of which literary competence levels teach- 


^ Arnon Grunberg, Willem Frederik Hermans and Harry Mulisch are well-known, canonical 
Dutch authors, while Ronald Giphart is very popular among students. 
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ers distinguish in practice. With this knowledge, we can lay an empirical foundation 
for a more structured curriculum for literature teaching. 

2. LITERARY DEVELOPMENT IN AN EDUCATIONAL CONTEXT 

In the context of education, literary development is primarily a socialization pro- 
cess (Garbe, 2009). Graf (1995) and Schbn (1995) have used reading biographies to 
conduct retrospective research into the literary socialization of children between 
the ages of two and eighteen. Garbe (2009) summarized some of the findings of 
their research. In the literary socialization process, she distinguishes intrinsic and 
extrinsic development factors and two critical development stages - primary and 
secondary literary initiation. Primary literary initiation occurs within the family, and 
secondary literary initiation takes place mainly in upper secondary education 
(grades 10-12). 

As well as internal development factors, such as brain function and socio- 
emotional and cognitive development in adolescents, there are extrinsic factors 
that play a key role in the literary development process (Alexander & Fox, 2011). 
Thus we saw above with students Rutger and Cecile just how powerful the influ- 
ence of the home environment can be. The teaching of literature also influences 
the literary development of students. Students are initiated into literature and cul- 
ture within the institutional environment of education (e.g. Hirsch, 1987; Purves & 
PradI, 2003; Beach et al., 2006, 2011). In upper secondary classes in particular, stu- 
dents are guided through new literary experiences. They often are stimulated, if 
not required to read texts that they would not have chosen themselves. They learn 
to share their reading experiences and interpretations with teachers and peers, 
acquiring certain rules and conventions of the genre of discourse (Beach & Mar- 
shall, 1991; Beach et al., 2006, 2011). In other words, the class is an 'interpretive 
community' (Fish, 1980) in which they develop their literary competence. 

2.1 Literary competence 

The term 'literary competence' comes from the literary theories of Culler (1975) 
and Schmidt (1980), who used it by analogy with Chomsky's linguistic competence. 
The term was used in various countries in relation to the objectives of literature 
teaching (Groeben & Vorderer 1988; Soetaert 1992). In the Netherlands the term 
occured in the late 1980s, in policy documents and professional journals for litera- 
ture teaching. 

The meaning of 'literary competence' became misty as a result of its frequent 
use as a new buzz word in a range of contexts; it threatened to become a 'catch-all 
term' (Ibsch & Schram, 1990: 20). For Coenen (1992) this was a reason to define 
the term for Dutch literature teaching. She did so by means of De Groot and 



Witte, Rijlaarsdam & Schram 


7 


Medendorp's (1986) 'semantic concept analysis', which resulted in the following 
definition: 

The literarily competent reader is able to communicate with and about litera- 
ture. The content of this communication can be highly diverse, but it does at least 
satisfy the requirement that the reader is able to construct coherence. This entails 
constructing coherence within a text to enhance comprehension, observing coher- 
ence and difference between texts, relating the text to the world (society and the 
personal world of the author) and relating personal judgement about the literary 
work to that of other readers. (...) The literarily competent reader's attitude to lit- 
erature is characterized by a willingness to invest in reading and to have an open 
mind with regard to unusual perspectives and frames of reference. (Coenen, 1992: 
73) 

This definition encompasses various parameters of literary competence which 
together describe the final attainment level of the typical student. We see these 
parameters in the 'mapping sentence' (Levy, 1976) with which Coenen operational- 
ized the concept of literary competence: 

A student (S) possesses a body of characteristics (C) which together constitute a iiter- 
ary competence (LC) that enabies that S to express particuiar views (V) about particu- 
iar texts (T) in a particuiar ianguage (L) at a particuiar iife stage (LS) and in particuiar 
situations (Si), views which - measured by particuiar means (M) and in accordance 
with particuiar criteria (Cr) - refiect performance at a particuiar ievei (Coenen, 1992: 

71). 

The adjective 'particular' expresses the notion that LS, L, T, V, M and Cr are varia- 
ble. These parameters thus offer clues for identifying different levels. The question 
is whether certain combinations of values on these parameters correspond to lev- 
els of literary competence as observed by teachers in their upper secondary clas- 
ses. We have therefore focused our research on the core of the mapping sentence, 
the interaction between student and text: what does a student with a particular 
level of literary competence (V, Cr) demonstrate with a particular literary text in 
upper secondary education (LS, L, Si, M)? 

2.2 Defining ieveis ofiiterary texts 

Basically we need to define the variety of literary texts and of student responses to 
this variety. Because reading novels dominates literary education in the Nether- 
lands, we focus on novels. Readability formulas like the Lexile measure are not 
suitable for our purpose as these assume that a text has an intrinsic level of difficul- 
ty irrespective of the attitude, skills and expectations that a reader brings to the 
text. Nystrand (1986) has criticized this belief in the 'autonomous text', arguing 
instead that 'difficulty' is as much a function of what readers bring to texts as the 
characteristics of the text itself (O'Brien et al., 2009). A second problem with Lexile 
measures is that the matches are based on very limited semantic and syntactic fea- 
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tures, like word frequency and average sentence length. Typical features of literary 
texts such as narrative structure, literary speech, implicitness, ambiguity are not 
taken into account. 

Jeanne Chall, who in 1958 was herself at the forefront of readability formulas, 
recognized the problem of validity. With her staff she developed a scale to qualita- 
tively assess text difficulty (Chall, Bissex, Conard & Harris-Sharples, 1996). For five 
dimensions in 15 grades, this scale describes what the reader needs to bring to the 
text in order to read literature with understanding. Table 1 shows what this entails 
for grades 10, 11 and 12. 

Table 1. What the reader needs to bring to read literature with understanding 
(Chall et al, 1996: 51) 


Knowledge of vocabulary 

Familiarity with sentence 
structures 

Depth and breadth of life 
experience 


Cultural and literary 
knowledge 

Skill in literary analysis 


(grades 7-12) 

Increasing number of uncommon words; non-literal meanings, 
(grades 9-15) 

Often complex and formal, currently uncommon structures 
appear in literature of earlier periods. 

(grades 5-15) 

Capacity for complex emotions and judgments, for reflection, 
for imaginative grasp of the inner lives of others. Especially at 
the higher levels of this range, the capacity to entertain unusual 
perspectives and multiple values. 

(grades 11-15) 

Cultural and literary knowledge increasingly essential for under- 
standing literature, especially from other periods and cultures, 
(grades 9-15) 

Literary analysis increasingly required for interpreting literature 
in academic contexts. 


At first sight this scale appears to satisfy our need to describe the interaction be- 
tween reader and text at different levels. Yet there are two problems that make it 
less suitable for our purposes. First, the scale does not describe features of texts 
but rather features of readers, while ignoring reader motivation and attitude. We 
agree with O'Brien et al. (2009) that educators need to take a realistic look at stu- 
dent engagement with reading at school. If students are not engaged, they cannot 
build proficiency. A second problem is that the classifications are very broad. As 
Table 1 shows, only one level is described for upper secondary education (grades 
10, 11, 12), whereas we are seeking to specify different levels of competence. 
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3. RESEARCH QUESTION AND RESEARCH DESIGN 

The core of our study is to define successively levels of literary competence in 
terms of the interaction between student and text on the base of shared pedagog- 
ical content knowledge of teachers. Three research questions guided our study, in 
which questions 1 and 2 are stepping stones to provide the answers on question 3. 

1. Which levels of literary competence do teachers globally distinguish in upper 
secondary education, varying from a poor reader at the start of grade 10 (gen- 
eral higher track) to a highly competent reader at the end of grade 12 (pre- 
university track)? 

2. Which texts do teachers consider indicative of a particular level? 

3. According to teachers, (a) which features do these indicative texts possess and 
(b) which attitude, knowledge and skills are called upon by these features at a 
particular level? 

3.1 Composition of the panei of teachers 

Much depends on the composition of the panel of teachers. For logistical reasons, 
the number of panel members had to be restricted to six. To make a weighted se- 
lection of respondents, we opted for a judgement sample of 'typical cases'. These 
are teachers who represent a commonly occurring type and not extreme deviations 
(Miles & Huberman, 1994; Flick, 2002). We aimed for the greatest possible varia- 
tion in terms of teaching experience (5 to 25 years), gender and philosophy of liter- 
ature teaching. For the latter, teachers were interviewed about the objectives that 
they pursued. Using Rijlaarsdam and Janssen's (1996) typology referred to above, 
we identified two groups: subject matter-oriented (cultural-historical and literary- 
aesthetic orientation) and student-oriented (social and personal orientation). We 
also looked for variation in school type (private, public) and region (village, town, 
city) because the day-to-day work setting helps shape a teacher's pedagogical con- 
tent knowledge (Shulman, 1986). Table 2 shows the general profile of the six 
teachers. 
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Table 2. General profile of the panel of teachers (names are fictitious) 



Teacher 

Subject 

matter- 

oriented 

Student- 

oriented 

Teaching 

experience 

(years) 

Work setting 
School 

Region 

Anna (1973) 

+ 

++ 

5 

state 

east, city 

Joan (1971) 

+++ 

++ 

5 

private 

north, city 

Sil (1960) 

+++ 

+ 

15 

private 

west, major city 

Laura (1950) 

++ 

++ 

23 

state 

north, village 

Ted (1947) 

+ 

++ 

28 

state 

east, village 

Joop (1949) 

+ 

+++ 

25 

private 

south, major city 


Table 3. Summary of the data-collectlon process 


Stage 1 


Stage 2 


Stage 3 


Aim 


Object 


Activity 


Result 


Distinguishing levels 
(question 1) 

Students in upper 
secondary education 


Work group: deter- 
mining number of 
levels (consensus) 


Competence scale 


Describing literary competences 
at different levels (question 2 
and 3) 

Teacher panel judgements 
(about level of books) and utter- 
ances (about features of books 
and literary competences of 
students) 

Question 2 

Individually: indicating level of 
books by means of question- 
naire 

Question 3 

Focus group: summarizing a 
panel discussion and categoriz- 
ing teacher statements in a data 
matrix 

Data matrix with literary compe- 
tences categorized by level (and 
graduated reading lists) 


Validation and synthe- 
sis 

Data matrix and com- 
petence profile 


Individually: evalua- 

tion of data matrix 
and competence pro- 
files 

Work group: deciding 
on data matrix and 
competence profiles 
(consensus) 

Validated data matrix 
and competence pro- 
files 
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3.2 Data collection and processing 

Data collection and processing occurred in three stages (see Table 3). Working con- 
ferences of two half-days were organized for each stage. The conferences were 
spread out over a longer period to minimize the pressure on panel members' time- 
tables. The first and third conferences took the form of a work group and the se- 
cond a focus group (Swanborn 1999; Flick 2002). A work group conducts a decision- 
making discussion while a focus group conducts an opinion-sharing discussion. The 
aim of a decision-making discussion was to reach a joint decision, whereas the 
opinion-sharing discussion was designed to collect as many opinions as possible 
from panel members. 

Stage 1 

The key question during the first working conference was which competence levels 
do teachers distinguish in upper secondary education? The panel designed a com- 
petence scale to distinguish the different levels of literary competence for grades 
10 and 11 of general higher education and grades 10, 11, 12 of pre-university edu- 
cation. The scale was to cover the range from a poor reader at the start of grade 10 
(general higher education) to a competent or highly competent reader at the end 
of grade 12 (pre-university). The panel distinguished six levels in total and assigned 
them general norms for both general higher education and pre-university educa- 
tion (see Table 4). 

Stage 2 

The second stage was devoted to formulating literary competences for the six lev- 
els in the competence scale. We first asked which books were indicative of a par- 
ticular level (question 2). We then examined the literary competences that books at 
a particular level call upon (question 3). This stage resulted in (a) lists of literary 
works that represent a particular level and (b) a data matrix containing levels and 
indicators in which the required literary competence could be described for each 
level. 

Competence levels indicated by literary works 

Using questionnaires we checked with the panel members which books were indic- 
ative of a particular level. For this purpose, we used the reading lists of the schools 
taking part in the study to compile a list of 170 novels, which we arranged random- 
ly using the alphabetical order. We designed a questionnaire for each of the six 
levels. Teachers rated all 170 books, starting with level 1. For each book they indi- 
cated whether it was: (1) too easy for that particular level, (2) easy for that level, 
(3) neither easy nor difficult for that level, (4) difficult for that level, or (5) too diffi- 
cult for that level. 
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The mean score indicates a book's optimum level. If the mean score is between 2.5 
and 3.5, the book is suitable for that level; if below 2.5 or above 3.5, the book is 
respectively too easy or too difficult for that level. The standard deviation of the 
scores of the six panel members indicates the degree of agreement (i.e. the shared 
pedagogical content knowledge). The smaller the standard deviation, the more 
respondents agree about a book's level, and the more reliable, we suggest, the 
level indication. 

Of itself, the mean is not a clear-cut indication of the degree to which a book 
represents a level. If a book A has a score of 3.0 at level N and got at level N+1 a 
score of 3.1, A is a less typical 'N book' than if book A got a score of 3.8 at level 
N+1. If in addition there was a marked difference in respondents' ratings (i.e. high 
standard deviations), the book's representativeness for level N is reduced still fur- 
ther. Effect size is a statistical measure for expressing what we here call represent- 
ativeness (Cohen, 1988) as it establishes the size of the difference between group 
scores. In the Results section we will show precisely how we determined the level 
of representation. 

Data matrix of levels of literary competence 

We were using a Delphi procedure to examine literary competences in qualitative 
terms. We opted for a panel discussion in the form of a focus group (Swanborn, 
1999) to elicit from the panel shared pedagogical content knowledge about six lev- 
els of literary competence. 

A discussion round of about 45 minutes was devoted to each level. The input 
for each round was a selection of 'critical cases' (Miles & Huberman 1994; Flick 
2002). Three groups of books were selected for a particular discussion round based 
on calculations of the mean score and effect size. These were (1) examples of 'high- 
ly representative books' (neither easy nor difficult for the level in question), (2) 
examples of 'borderline cases' (slightly easy or slightly difficult), and (3) 'counter 
examples' (much too easy or much too difficult). 

For each case, the panel was asked to discuss why a book was representative of 
the level in question and why another book was a little or much too easy or diffi- 
cult. The facilitator ensured that all members had sufficient opportunity to speak 
and the discussion remained on track, focusing on the relationship between reader 
and text. The facilitator also encouraged participants to express themselves in as 
concrete a fashion as possible so that the results would be meaningful to other 
teachers (Swanborn 1999). 

The respondents' views were recorded as literally as possible by three minute 
keepers. Audio recordings were also made of both conferences so that the minuted 
comments could be verified later. For each level this gave us a set of comments 
(pedagogical content knowledge) about what a student shows in terms of literary 
competence with texts of a particular level. 
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To analyse this data we used an empirical, interpretive research methodology 
(Miles & Huberman, 1994; Denzin & Lincoln, 1994). Without making use of an a 
priori system, this methodology seeks to discover certain regularities in the data 
and to reduce them step by step to dimensions of variables. The analyses were 
performed using ATLAS. ti. We firstly labelled the minuted teacher comments and 
then sorted the labelled comments into levels and parameters (student, text), and 
grouped the labels into 'families' (e.g. 'willingness', 'knowledge of the world', 'fa- 
miliarity with literary techniques'). These families gradually built up the dimensions 
and indicators of literary competence with which to systematically describe the 
different levels. The analyses resulted in a data matrix, with the rows representing 
the dimensions and indicators of literary competence and the columns the levels of 
literary competence. Lastly, the respondents' comments were reformulated into 
succinct and distinctive descriptions. 

Stage 3 

At the last working conference the panel validated the data matrix containing the 
reformulated comments. An important principle here was that the results should 
be based on shared pedagogical content knowledge. 

As a preparation for the working group session and to maximize the validity of 
the competence descriptions, we submitted the data matrix to the six panel mem- 
bers and to six external experts (two literary theorists, two teaching methodolo- 
gists and two teachers). The respondents were sent the minutes of the discussions 
and the data matrix and were asked to check the correspondence between the 
comments and the matrix and to make concrete suggestions for changes in the 
matrix. 

These suggestions formed the input for the third working conference with the 
six panel members. Consensus was reached on each point in the discussion of the 
suggested changes. Finally, for each level we synthesized the descriptions in the 
different dimensions and indicators into 'literary competence profiles'. These pro- 
files were also validated by the panel members and experts. 

4. RESULTS 

The results consist of a competence scale (4.1), indicated book lists (4.2), a data 
matrix (Appendix 1) and lastly, a description of six successive literary competence 
levels (4.3). 

4.1 Competence scale 

Before our investigation into competence levels and the operationalization of the 
text parameter, we asked the following question: how many levels of literary com- 
petence do teachers distinguish in upper secondary classes? According to teachers. 
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the levels should be sufficiently distinctive, both for themselves and the students. 
The teachers agreed on six levels for upper secondary, working from the premise 
that the starting levels of students in year 4 can vary enormously because of the 
broad range of programmes in lower secondary schools (an ill-structured curricu- 
lum!). Fairly neutral labels were preferred for the initial, general descriptions of the 
levels: (1) very limited, (2) limited, (3) neither limited nor extended, (4) fairly broad, 
(5) extended and (6) very extended literary competence. These are of course rela- 
tive descriptions which in principle only apply to students in upper secondary clas- 
ses. To make these descriptions concrete and to place them in the context of de- 
velopment over several years, they were given a label that was meaningful for 
teachers. For example, level 3 is a 'good' performance at the start of literature edu- 
cation in grade 10 and a 'satisfactory' performance at the end of grade 11, but it is 
an 'unsatisfactory' performance at the end of grade 12 (see Table 4). 
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Table 4. Competence scale with common benchmarks and norms for six levels of literary competence for general higher education (HAVO) and pre- 

unlverslty (VWO) 


Level 1 

Level 2 

Level 3 

Level 4 

Level 5 

Level 6 

Very limited 

Limited 

Neither limited 

Fairly broad compe- 

Extended compe- 

Very extended 

competence 

competence 

nor extended 

tence 

tence 

competence 


competence 


Complexity 


Starting 
level 
grade 10 
Final level 
grades 
11/12 


Cannot read, under- 
stand or appreciate 
very simple literary 
works 
Weak 


Very poor 
(HAVO: < 4) 


Can read, under- 
stand and appreci- 
ate very simple 
literary works 
Reasonable 


Poor (HAVO: 5) 
Very poor 
(VWO: <4) 


Can read, under- 
stand and appreci- 
ate simple literary 
works 
Good 


Can read, understand 
and appreciate literary 
works of a medium 
level of difficulty 
Very good 


Can read, under- 
stand and appreci- 
ate complex liter- 
ary works 
n/a 


Can read, under- 
stand and appreci- 
ate very complex 
literary works 
n/a 


Satisfactory 
(HAVO: 6 to 7), 
Poor (VWO: 5) 


Good 

(HAVO: 8 to 9), 
Satisfactory (VWO: 6 
to 7) 


Very good (HAVO: Very good 

9tolO)Good (VWO: 9 to 10) 

(VWO: 8 to 9) 


Note. The numbers in the last row refer to the mark the teachers would have given for performance in the final examination, whereby 5 is ‘fail’, 6 is ‘pass’, 7 is 
‘satisfactory’, 8 is ‘good’, 9 is ‘very good’ and fO is ‘excellent’. HAVO refers to the general higher education track, and VWO to the pre-university track. 
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4.2 Classification of books into six levels 

We received questionnaires from all six respondents (36 in total). The level could 
not be reliably established for 29 books because too few teachers were familiar 
enough with those books. The respondents proved a reliable jury: for all six levels 
the homogeneity between the respondents was high for 141 literary works 
(Cronbach's alphas for the respective levels were .89, .92, .91, .91, .88 and .78). We 
were able to reliably establish the level of these books: four were appropriate for 

level 1, 11 for level 2, 43 for level 3, 68 for level 4, 29 for level 5 and two for level 6 
2 

(see Figure 1). 



Levels 

Figure 1. Number of books per level. Tbe x-axis shows tbe competence levels and the y-axis 
the number of books. (NB. The total number of indications is 157. This is because some works 
(N=16) were seen as indicative of two levels and therefore occur twice in the data. 

Table 5 shows an example of the reading list for level 2, the desired starting level 
for upper secondary classes (see Table 3, grade 10). The effect size indicates the 
extent to which a book is representative of that level. We applied the following 
standards: .2 indicates a small effect, .5 a medium effect and .8 a large effect (Co- 
hen, 1988: 28). The smaller the effect, the less the literary work is representative of 
a level. We therefore linked these standards to three gradations of representatives, 
whereby A = highly representative of the level (large effect > .8), B = representative 


2 The small number at levels 1 and 2 and level 6 shows that the middle group (levels 3, 4 and 
5) has a wide range to choose from, but that the range for the two lowest and the highest 
level is very small. This illustrates our comment above that Dutch teachers concentrate on 
the middle group and do not cater sufficiently for the weakest and strongest students. 


of the level (medium effect > .5), and C = somewhat representative of the level 
(small effect > .2). 


Table 5. Overview of books with an L2 indication ranked by mean complexity (M). if M is < 
2.5, the book is easy/too easy for level 2; if M is > 3.5, the book is difficult/too difficult for 
level 2. SD is the standard deviation of the teacher ratings. R indicates representatives, with 
A showing that the book is 'very representative' of level 2, B 'representative' and C 'fairly 

representative 




LI 


L2 


L3 


R 

Literary Work 

M 

SD 

M 

SD 

M 

SD 

A 

Keuls, Jan Rap en z'n maat 

3.2 

.41 

2.8 

.41 

2.0 

.63 

A 

Dragt, Torens van februari 

3.5 

.58 

2.8 

.50 

2.0 

.00 

A 

Ruyslinck, Wierook en tranen 

3.7 

.52 

2.8 

.41 

2.3 

.52 

A 

Sahar, Hoezo bloedmooi? 

3.3 

.58 

3.0 

.00 

2.0 

.00 

A 

Van Lieshout, Gebr. 

3.3 

.58 

3.0 

.00 

2.3 

.58 

A 

Oberski, Kinderjaren 

3.5 

.55 

3.0 

.00 

2.3 

.52 

C 

Btlch, Het doihuis 

4.7 

.52 

3.0 

.52 

2.8 

.41 

A 

Frank, Het achterhuis 

3.7 

.52 

3.0 

.63 

2.3 

.82 

B 

De Loo, Isabelle 

4.2 

.75 

3.2 

.75 

2.7 

.82 

A 

Giphart, De voorzitter 

4.0 

.00 

3.3 

.50 

2.3 

.50 

A 

El Bezaz, De weg naar het noorden 

4.3 

.58 

3.3 

.58 

2.7 

.58 


Twelve of the 170 books were given an L2 indication, with Jan Rap en z'n maat the 
easiest and De weg naar het noorden the most difficult book in the list. Hoezo 
bloedmooi, Gebr. and Kinderjaren are the most representative for this level (me = 
3.0 and SD = .00). In this list we also see that the level indication is not absolute, 
with most books matching two levels: Jan Rap en z'n maat, Torens van februari, 
Hoezo bloedmooi, Gebr. and Kinderjaren match LI and L2, and Het doihuis, Isabelle 
and De weg naar het noorden match L2 and L3. 

4.3 Description of six levels of literary competence 

As outlined above in Section 3.2 (stage 2), the books classified by level formed the 
basis for the panel discussions about which literary competences students need in 
order to independently understand books of a particular level. Once again, the 
panel proved very homogeneous. In their discussions they seldom disagreed about 
whether or not students with a particular level of literary competence could inde- 
pendently understand books of a particular level of difficulty. For example, if dis- 
cussing whether a grade 10 student with a very limited literary competence (level 
1) could read and understand Harry Mulisch's De aanslag (level 4), they came to 
similar analyses and conclusions. It struck us that the panel members hardly chal- 
lenged one another's views, but supplemented them by citing their own experienc- 




es and examples from their own teaching practice. These discussions resulted in a 
literary competence matrix and ultimately in six competence profiles. 

Tying in with Coenen's mapping sentence (see 2.1), the matrix contains two pa- 
rameters with which to describe literary competence at six levels: (1) the student 
parameter describes the disposition of the student as reader; (2) the text parame- 
ter describes the competences that relate to certain features of the text. The text 
parameter comprises three dimensions of literary competence: familiarity with 
literary style, familiarity with literary techniques, and familiarity with literary char- 
acters. 

Each dimension in turn comprises indicators of literary competence, fourteen in 
total. These are obviously the variables that teachers use to determine the level of 
literary competence of students in upper secondary education. In Table 6, we give 
an overview for each parameter of the dimensions and indicators, with a brief de- 
scription of the complicating factors. 
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Table 6. Overview of dimensions and indicators of literary competence 


Dimension Indicator Description (complicating factors) 


STUDENT 
General de- 
mands for 
engaging 
with the 
book 


Willingness 

(time) 

General 

knowledge 

Domain- 
specific (liter- 
ary) 

knowledge 
and experi- 
ence 


The extent to which the length of the text (number of pag- 
es) requires a certain investment of time. 

The extent to which the text calls for general knowledge, 
which is world knowledge (societal, historical) and anthro- 
pological knowledge (intercultural, social, psychological). 
The extent to which the text calls for experience with liter- 
ature and domain-specific knowledge, such as the history 
of literature (literary periods), literary theory and intertex- 
tuality. 


BOOK 

Familiarity Vocabulary 

with literary 

style 


Sentence 

structure 


Stylistics 


The extent to which the text calls for a certain familiarity 
with particular registers of language use, such as the level 
of abstractness, the proximity of the represented world, 
and the diversity of vocabulary (archaic or regional varie- 
ties), possibly because of a historical distance (non- 
contemporary literature). 

The extent to which the text calls for a certain level of 
familiarity with fairly complicated sentence structures 
(length, embedding, sequence of meaningful elements), 
possibly because of a historical distance (non- 
contemporary literature). 

The extent to which the text calls for knowledge of literary 
language use and its changes over time, that is, the extent 
to which language is figurative, multi-interpretable and 
refers to conventions and stylistics. 


Familiarity Act 
with literary 
techniques 


The extent to which the text holds the reader's attention 
(suspense). This includes the pace of action and the se- 
quence and intensity of dramatic events. It also includes 
story elements that interrupt the course of action and 
complicate the reading (internal monologues, reflections, 
descriptions, elaborations and expositions). 



Chronology The extent to which the text demands flexibility with re- 
spect to the chronology and continuity of the action. Shifts 
in time, references to the past (flashbacks) and references 
to the future all complicate the reading process. 

Storyline(s) The extent to which the text calls on the ability to simulta- 
neously follow different storylines and link them to each 
other. The number of strands and the links between them 
(primary, secondary, embedded) influence the level of 
complexity. 

Perspective The extent to which the text calls on the ability to distin- 
guish between different perspectives. The reliability of a 
perspective and how this can be played out (manipulation) 
forms an additional complicating factor. The first-person 
narrative used throughout the text is also accessible be- 
cause the reader has only one centre of focus within the 
fictional world. In general, changes in perspective are a 
complicating factor (multiple perspectives). An omniscient 
narrator is generally considered less complex because of 
his or her intermediary status between reader and story. 

Meaning The extent to which a text requires the ability to recognize 

and connect various levels and elements of meaning (e.g. 
irony and parody). Complexity increases with the number 
of levels (reality, psychological, political, philosophical, 
literary, etc.) and elements (motifs, themes, ideas) that are 
included. A high degree of implicit information also makes 
the reading more complex. 


Familiarity Character The extent to which a text calls for the ability to fathom 

with literary both character and character development. It refers to the 

characters level of characterization as well as character development 

(type and character). Further complicating factors are the 
level of predictability/unpredictability and the reader's 
distance from the morals and behaviour of the characters, 
their historical status (old texts) or level of abstraction 
(literary persona) 

Number The extent to which a text calls for the ability to differenti- 

ate between main and subsidiary characters. The number 
of characters is a complicating factor. 

Relations The extent to which a text calls for the ability to fathom 

the relationships between characters. The nature of the 
relationships (psychological, sociological, intercultural) and 
any related changes in them are complicating factors. 



If we combine the six levels with the 14 indicators, we obtain a system to describe 
the literary competence for all six levels. Appendix 1 presents an abbreviated ver- 
sion of the matrix. The cells in the matrix provided the building blocks for a more 
holistic description of each of the six levels of literary competence in a competence 
profile. Once the six competence profiles had been validated, each profile was giv- 
en a descriptive label denoting a particular kind of reading. 

Profile level 1: Experiential reading (very limited literary competence) 

Student Students with very limited literary competence have little experience of 
as read- reading fiction. They have difficulty reading, understanding, interpreting 
er and appreciating very simple literary texts, as well as communicating 

about their reading experiences and reading tastes. Their general level 
of development is inadequate for gaining entry to the world of literary 
books for adults. They are unfavourably disposed towards literature 
because they find the content too remote and the style too difficult. 
Their willingness to invest in literature is slight. Book size and task type 
are therefore factors that weigh heavily with these students. Their ideas 
about literature and their attitude to reading are characterized by a 
need for tension (action) and drama (emotion). Their kind of reading 
can be labelled experiential reading. 

Text The books suitable for these students are written in simple, everyday 
language and are closely linked to the experiences of adolescents in 
terms of content and characters. The storyline is clear and simple, with 
exciting or dramatic events succeeding one another at a rapid pace. 
There are few structural elements, such as thoughts or descriptions, to 
interrupt the action. 

Profile level 2: Identifying reading (limited literary competence) 

Student Students with limited literary competence have experience of reading 
as read- fiction, but almost none of reading literary novels for adults. They are 
er capable of reading, understanding and appreciating very simple literary 

texts and can report on their personal reading experience and tastes. 
Their general level of development is sufficient to understand adult 
literature, but not to gain entry to a reality, or 'novel world', that is very 
different from their own experience. They are willing to invest in litera- 
ture, but not to a great degree. As a result, book size and task continue 
to guide their choice of book. Their attitude to reading is characterized 
by an interest in recognizable situations, events and emotions. They 
believe that literature should be realistic. This kind of reading can be 
labelled identifying reading. 

Text The books suitable for these students are written in everyday language, 
have a simple structure and relate to their experiences. Although the 
books are written for adults, the main character is usually an adoles- 



cent. There is a dramatic storyline in which actions and events succeed 
one another at a fairly rapid pace. It is not particularly problematical if 
the tension is interrupted from time to time by thoughts and descrip- 
tions. These students prefer a closed ending. 

Profile level 3: Reflective reading (somewhat limited literary competence) 

Student Students with somewhat limited literary competence have experience 
as read- of reading simple literary texts. They are capable of understanding, 
er interpreting and appreciating simple literary works and can discuss with 

classmates social, psychological and moral issues based on a book. 
Their general and literary development is sufficient to gain entry into a 
somewhat complex novel structure and into the world of adults. They 
are willing to invest in literature, but will not readily embark on a thick 
book or a more complex task. Their attitude to reading is characterized 
by an interest in social, psychological and moral issues. For them, litera- 
ture is a means of exploring the world and forming their own ideas on a 
wide range of issues. Reading at this level can be labelled reflective 
reading. 

Text The books suitable for these students are written in simple language 
and have a complex but nonetheless transparent structure with a 
deeper layer of meaning alongside the concrete one. The content and 
characters do not relate directly to the experience of adolescents, but 
the story addresses issues that interest them, such as love, death, 
friendship, justice and responsibility. Their preference is for texts deal- 
ing with social or political issues. Inasmuch as the text contains complex 
narrative techniques (such as shifts in time, changes in perspective, 
motifs, etc.), these tend to be explicit. The story confronts the reader 
with questions that may remain unanswered and usually has an open 
ending. 

Profile level 4: Interpretive reading (fairly broad literary competence) 

Student Students with a fairly broad literary competence have experience of 

as read- reading simple literary novels for adults. They are capable of reading, 

er understanding, interpreting and appreciating literature that is not too 

complex and they can communicate effectively about their interpreta- 
tions and tastes. Their general and literary development is sufficient to 
allow them to gain entry to the novels of notable literary authors, pro- 
vided these are not too complex. They are clearly willing to invest in 
literature. The number of pages and the size of the task are no longer 
so relevant. These students display a budding aesthetic awareness: 
they are discovering that a literary novel is 'created' and that writing is 
an 'art' and not a 'trick'. These students' attitude to reading is charac- 
terized by a willingness to immerse themselves in complex events and 



adult emotions that are far removed from their own experience. They 
are interested in narrative technique and novel structure, and perhaps 
also in the author's intent. Reading at this level can be labelled inter- 
pretive reading. 

Text The books suitable for students at this level are written in a 'literary' 
style and are not immediately related to the world of adolescents in 
terms of content and characters. This makes the storyline and character 
development less predictable. The literary techniques used are some- 
what complex: unreliable perspective, implicit time shifts and changes 
in perspective, unanswered questions, multiple layers of meaning, 
metaphorical style, etc. These techniques encourage the reader to in- 
terpret the text. At this level we encounter many well-known works by 
notable authors. 

Profile level 5: Literate reading (extended literary competence) 

Student Students with extended literary competence have broad experience of 
as read- reading literary novels. They are able to understand, interpret and ap- 

er predate complex works, including old literary texts written before 

1880, as well as to exchange ideas with others about their reading ex- 
perience, interpretation and tastes. Their general, historical and literary 
knowledge is sufficiently advanced to gain entry to complex modern 
and old classical texts. They are willing to read these texts and to im- 
merse themselves not only in themes and structure, but also in literary- 
historical backgrounds and style. They are aware that texts operate 
within a cultural and historical context and that literature is a tool for 
learning about the past and about cultural identity. Their attitude to 
reading is characterized by an interest in the canon, literary conven- 
tions, cultural and historical backgrounds and certain classical authors. 
This kind of reading can be labelled literate reading. 

Text The books suitable for students at this level not only contain characters 
and themes that are far removed from their own experience, but may 
also differ greatly from what they are accustomed to in terms of lan- 
guage use and literary conventions. This is particularly true of old texts, 
which contain a historical novel reality with obsolete values and norms, 
and of old Dutch, with its outmoded literary conventions. With modern 
novels, we observe an increase in the complexity of the novel structure, 
which is characterized by ambiguities and implicitness, as well as by 
technical and stylistic refinements. 

Profile level 6: Academic reading (very extended literary competence) 

Student Students with an academic literary competence have a wealth of expe- 
as read- rience of reading literary texts, including world literature. They are able 

er to place books and literature in a broad context and to discuss their 



reading experience and interpretations with 'experts'. Because they are 
so well read and possess both highly developed general knowledge and 
specific cultural and literary knowledge, they are able to establish links 
and to generate meanings both within and beyond the text. They have 
a great willingness to invest in literature, provided they have some de- 
gree of autonomy. They are of the view that literature adds a further 
dimension to their lives and helps them understand reality (existential 
function). They have a critical approach to reading, characterized by 
versatility, passion and an interest in specialized literature on the sub- 
ject. This kind of reading can be labelled academic reading. 

Text The texts suitable for these students are written in a fairly inaccessible 
literary style which may include experimental forms and styles. They 
have a multi-layered, complex structure, making it difficult to penetrate 
through to the story and to interpret its meaning. The text has symbolic 
features (abstract motifs) and contains references to other texts and 
knowledge (intertextuality) that are essential for a proper understand- 
ing. 


5. CONCLUSION AND DISCUSSION 

The aim of this study was to create a frame of reference for the development of 
literary competence in the context of literature teaching in upper secondary educa- 
tion. Such a frame of reference should make it easier to identify differences be- 
tween students and to deliver teaching that stimulates the literary development 
process of students with different levels. A key principle for us was to base the 
frame of reference on the shared pedagogical content knowledge of a varied group 
of teachers so that the research outcomes would be meaningful for teachers. 

The result is that we identified six levels of literary competence that relate to 
norms for general higher education and pre-university education (Table 1). We 
used two parameters (student and text) taken from Coenen's definition of literary 
competence to operationalize six levels of literary competence. For pragmatic rea- 
sons we decided to restrict ourselves to the novel. This resulted among other 
things in reliable graduated reading lists (Table 4 and Figure 1), an instrument to 
describe the difficulty and accessibility of texts for different reading levels (Table 6) 
and a data matrix and six competence profiles in which six consecutive levels of 
literary competence are described based on fourteen dimensions and indicators. 
Both the competence profiles and the data matrix used to compile the profiles 
were validated by the panel of teachers and by a group of external experts. Inas- 
much as we have been able to ascertain, this is the first time that the link between 
graduated reading levels for students aged 15 to 18 and graduated levels of diffi- 
culty for literary texts has been described so systematically and in such detail. 


Consistency 



The fact that we were able to describe the competences for all six levels in the data 
matrix means that there is consistency both within and between levels. The panel 
identified the consistency within levels as a particular kind of reading: 'experiential' 
(level 1), 'identifying' (level 2), 'reflective' (level 3), 'interpretive' (level 4), 'literate' 
(level 5) and 'academic' (level 6). Each level clearly focuses on a particular function 
of literature and the acquisition of the associated kind of reading: you can read for 
pleasure (level 1), recognize your own experience and find self-affirmation (level 2), 
expand your horizons (level 3), discover deeper meanings and aesthetic enjoyment 
(level 4), immerse yourself in literature, culture and history (level 5), and nourish 
your intellect (level 6). 

The six levels of literary competence appear to reflect a development spiral 
(Bruner, 1964, 1984; Kohiberg, 1969; Van Parreren, 1988). How this development 
progresses needs to be further investigated. Given the changes in reasons for read- 
ing and in the view of literature at each level, this may not be a gradual and contin- 
uous process but rather a discontinuous process that occurs in fits and starts, 
whereby each stage lays the foundation for the next, with new structures being 
integrated into existing ones (Piaget, 1952). In that case, the six competence pro- 
files could be seen as repertoires of mental operations that a student can apply 
flexibly. Flexibility is thus a feature of a higher level of literary competence. A read- 
er with, say, a fairly broad literary competence (level 4) is able to use different 
reading modes (experiential and/or identifying and/or reflective and/or interpre- 
tive), whereas a reader with a very limited literary competence (level 1) has only 
one reading mode - a literary text should entertain. Literary development can pos- 
sibly be regarded as a cumulative process in which students expand their repertoire 
of reading modes step by step and in which literary texts acquire different func- 
tions. 

Reliability 

Important questions are (1) whether the graduated reading lists and the six compe- 
tence profiles reliably reflect the stages of literary development and (2) whether 
the instrument is ecologically and theoretically valid. In principle, the small number 
of respondents limits the generalizability of our findings. Parts of this study need to 
be replicated in order to determine whether our results are valid. Nonetheless, we 
feel that there are already strong indications that the levels of literary competence 
that we have identified are ecologically valid. In order to generalize and formalize 
the knowledge basis, this must be based on teachers' shared pedagogical content 
knowledge (Verloop, 2003), hence our decision to opt for six teachers, both male 
and female, who represented a broad range of views, experience and work set- 
tings. However, these differences had little effect on the ratings for the relative 
complexity of books. There was shared pedagogical content knowledge in both the 
individual ratings and the focus group discussions about critical cases. This was 
even more the case in the validation of the data matrix and competence profiles 
(stage 3) where, apart from a few style issues, both the teacher panel and the ex- 
pert group confirmed the study results 



To be able to generalize our findings to other educational contexts, the study 
needs to be replicated in other countries. In 2010, as part of a European project, a 
five-country study was launched into levels of literary competence for 12 to 18- 
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year-olds. Preliminary findings show that teacher panels in other countries come 
to the same conclusions as the Dutch panel, which suggests that different groups of 
teachers in different contexts are likely to arrive at similar conclusions and that a 
generalizable development model is quite plausible. 

With regard to generalizability, we can also ask about the extent to which this 
classification has theoretical validity. Anglo-Saxon literature in particular is con- 
cerned with the theory of stages of literary development. Applebee (1978), Thom- 
son (1987) and Appleyard (1994) have researched the literary development of vari- 
ous groups of subjects. In the Netherlands, Neick-da Silva Rosa and Schlundt 
Bodien (2004) investigated the development of students' reading attitudes in rela- 
tion to Loevinger's (1976) stages of ego development. All these developmental 
theories arrive at five or six virtually identical development stages. There are also 
strong parallels with the phasing in aesthetic development models (Housen, 1983; 
Parsons, 1987; Gardner, 1990). Although the models were developed in different 
contexts using different study samples, a range of research methods and different 
research objects, they nevertheless have an identical phasing. According to Gard- 
ner, 'such a consensus is rare in the behavioral sciences and strongly suggests that 
we have here an instance of genuine developmental sequence' (Gardner, 1990: 
17). This means that the instrument is supported not just by the practical 
knowledge of teachers, but by developmental theory as well. 

Our research methodology using a fairly small but varied group of teachers has 
clearly produced reliable outcomes for the development of formalized, generaliza- 
ble pedagogical content knowledge about the relationship between different levels 
of mastery and the difficulty of learning tasks. This is an interesting observation for 
this type of pedagogical study. Replication studies in other domains of mother- 
tongue education and with other school subjects will have to demonstrate the ex- 
tent to which our research methodology can be extrapolated. 

In conclusion 

A rigid pedagogical application of our instrument in practice, as is still the case with 
the classical Lexile measures, would be entirely wrong in our view. The primary 
function of the competence profiles and graduated reading lists within teaching is a 
heuristic one. A teacher who can work with this frame of reference 'sees and 
knows more' and can therefore deliver appropriate instruction to students with 
different starting competences, like Rutger and Cecile in the introduction to this 


^ Comenius Project: 'Literary Framework for Teachers in Secondary Education' (LiFT-2). This 
project is part of the European Union's Lifeiong Learning Programme. In addition to the 
Netherlands, the countries taking part in this project are the Czech Republic, Finland, Germa- 
ny, Portugal and Romania. The first results will be available in 2012 on 
www.literaryframework.eu. 



article. Members of the teacher panel reported one year later that they had inter- 
nalized the frame of reference and were increasingly starting to think 'in levels'. 
Preliminary student experiences with the instruments show that the frame of ref- 
erence also provides guidance for students. They can recognize themselves and 
their classmates in the profiles and can note the direction in which to develop their 
literary competence. Initial reports also show that many students find it a challenge 
to aim at higher reading levels. For teaching teams and policymakers too, the in- 
strument can serve as a frame of reference because they can use it to underpin 

4 

agreements on matters such as the desired level of a particular group. For this 
reason the instrument also offers a frame of reference for designing a structured 
curriculum. 

The instrument provides researchers with a 'yardstick' for mapping the literary 
development of various groups of students. This implies in principle that it can be 
used to evaluate the outcomes of literature teaching and to substantiate, using 
empirical arguments, pronouncements on such matters as the progress or regres- 
sion of particular groups of students - in principle as the instrument is not yet 
complete. We have yet to operationalize the pedagogical variables 'tasks' and 'ef- 
fects' in Coenen's mapping sentence. We will be reporting on this in a subsequent 
article that will focus on the question of how to stimulate literary development. 
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Data matrix: Teachers' pedagogical content knowledge about levels of literary competence in upper 
secondary education. Level 3 is grey-shaded 
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